Multilevel Analysis in CSCL Research

S. Puntambekar et al. (eds.), Analyzing Interactions in CSCL, Computer-Supported Collaborative Learning Series, DOI 10.1007/978-1-4419-7710-6_9, © Springer Science+Business Media, LLC 2011 Abstract The aim of this chapter is to explain why multilevel analysis (MLA) is often necessary to correctly answer the questions CSCL researchers address. Although CSCL researchers continue to use statistical techniques such as analysis of variance or regression analysis, their datasets are often not suited for these techniques. The first reason is that CSCL research deals with individuals collaborating in groups, often creating hierarchically nested datasets. This means that such datasets for example contain variables measured at the level of the individual (e.g., learning performance) and variables measured at the level of the group (e.g., group composition or group performance). The number of unique observations at the lowest level, the individual, is higher than at the highest level, the group. Related to this, CSCL datasets often contain differing units of analysis. Some variables that CSCL researchers are interested in are measured at the individual level (e.g., gender, interactive behavior, familiarity with other group members), whereas other variables are measured at the group level (e.g., gender group composition, group performance, group consensus). Finally, because group members interact with each other in CSCL environments, this leads to nonindependence of the dependent variable(s) in the dataset. Because of their common experience during the collaboration, students’ scores on the dependent variables will likely correlate (e.g., in a group with a relatively long history of successful collaboration, group members will report similar, high levels of trust, while in groups with a negative collaboration history, group members will likely report low levels of trust). Whether nonindependence is present in a dataset can be established by calculating the intraclass correlation coefficient. Whenever researchers encounter datasets with hierarchically nested data, differing units of analysis, and nonindependence, MLA is needed to appropriately model this data structure since it can appropriately disentangle the effects of the different levels on the dependent variable(s) of interest. Researchers however also employ other strategies to deal with nonindependence and hierarchy in their datasets (e.g., ignoring nonindependence and hierarchy, or